Effective and Efficient Transfer Learning in the Era of Large Language Models

Substantial progress has been made in the field of natural language processing (NLP) due to the advent of large language models (LLMs)—deep neural networks with millions or billions of parameters pre-trained on large amounts of unlabeled data. However, these models have common weaknesses, including degenerate performance in data-scarce scenarios, and substantial computational resource requirements. This thesis aims to develop methods to address these limitations for improved applicability and performance of LLMs in resource-constrained settings with limited data and/or computational resources. To address the need for labeled data in data-scarce scenarios, I present two methods, in Chapter 2 and Chapter 3, respectively. The first method leverages beneficial relationships between NLP tasks for transfer learning, while the second method combines data augmentation and self-training to boost few-shot learning performance—the ability to perform novel tasks from only a few labeled examples. Additionally, in Chapter 4, I introduce a novel parameter-efficient transfer learning approach that reuses a single frozen model for all tasks while only learning minimal task-specific parameters (soft/continuous prompts) to represent tasks and transfer knowledge. Our method can match or outperform fine-tuning task-specific models (training the whole model on each task). In Chapter 5, I demonstrate the benefits of parameter-efficient transfer learning in a cross-lingual transfer setting. Finally, I conclude the thesis in Chapter 6 by outlining potential avenues for future research that aim to advance NLP through large-scale multi-task learning using multilingual and multimodal data

A car lane-changing model under bus priority-lane effects

Car lane-changing behaviour has been well investigated at merging locations or weaving sections where the lane-changes are usually due to different origin-destination trip purposes. However, the lane-changing behaviour under the effects of bus priority-lanes in urban streets has not been received much attention. This kind of behaviour is found to initially depend on the existence of oncoming buses in priority-lanes in urban streets. In this paper, a car lane-changing model under bus priority-lane effects in urban streets is proposed. This model comprises three steps: looking-back threshold determination, gap acceptance model and execution model. The model’s parameters are estimated jointly by using the Maximum Likelihood Method. The research results show that the car lane-changing behaviour under bus-priority-lane effects in urban streets is considered compulsory behaviour. The behaviour has specific characteristics with smaller critical gaps compared with those at other normal lane cases and can be modelled by the proposed model

Zooplankton Composition in Super-Intensive Whiteleg Shrimp, Litopenaeus vannamei (Boone, 1931) Culture Tanks

This study aimed to determine the zooplankton species composition in super-intensive whiteleg shrimp, Litopenaeus vannamei (Boone, 1931) tanks. The research was conducted from January to May 2021 in Bac Lieu City, Bac Lieu province, Vietnam. Eleven sampling times were divided into two periods, the nursery phase (six times) and the grow-out phase (five times) of shrimp culture. The results showed that water quality parameters fluctuated dramatically during the culture period, in which some nutrient concentrations tended to increase at the end of the shrimp culture period. Nine zooplankton species were recorded, of which five species belonged to Protozoa, three Rotifera species, and one Copepoda species. The number of zooplankton species did not differ significantly among the sampling periods. Protozoa had the highest species composition and density during most of the shrimp culture period. Copepoda was only identified in the nursery stage of shrimp culture. The species component of zooplankton had a close positive correlation with temperature, but their abundance did not have a significant relationship with water quality parameters because each species was affected by the different water quality parameters. Zoothamnium sp. had significantly positive correlations with total ammonia nitrogen (TAN), nitrate (NO3)-, total phosphorus (TP), and total nitrogen (TN) concentrations. The rotifer Brachionus plicatilis had a strong relationship with TP content, whereas Dartintinnus alderae had a strong relationship with alkalinity. Protozoa dominance in shrimp tanks could affect shrimp growth, decreasing the economic efficiency of shrimp farming. Therefore, the results of this study contribute to water quality and natural food management to improve shrimp productivity